DEFENSE  DISBURSING  YEAR  2000  END-TO-END  TESTING 


Report  No.  00-006  October  12,  1999 

Office  of  the  Inspector  General 
Department  of  Defense 


DISTRIBUTION  STATEMENT  A 

Approved  for  Public  Release 
Distribution  Unlimited 

jjHOQUALiarsf 


20000210  012 

JT0£6O^'3' 


Additional  Copies 

To  obtain  additional  copies  of  this  audit  report,  contact  the  Secondary  Reports 
Distribution  Unit  of  the  Audit  Followup  and  Technical  Support  Directorate  at 
(703)  604-8937  (DSN  664-8937)  or  fax  (703)  604-8932  or  visit  the  Inspector 
General,  DoD,  Home  Page  at;  www.dodig.osd.mil. 

Suggestions  for  Future  Audits 

To  suggest  ideas  for  or  to  request  audits,  contact  the  Audit  Followup  and 
Technical  Support  Directorate  at  (703)  604-8940  (DSN  664-8940)  or 
fax  (703)  604-8932.  Ideas  and  requests  can  also  be  mailed  to: 

OAIG-AUD  (ATTN;  AFTS  Audit  Suggestions) 

Inspector  (jeneral.  Department  of  Defense 
400  Army  Navy  Drive  (Room  801) 

Arlington,  VA  22202-2884 

Defense  Hotline 

To  report  fraud,  waste,  or  abuse,  contact  the  Defense  Hotline  by  calling 
(800)  424-^98;  by  sending  an  electronic  message  to  Hotline®dodig. osd.mil;  or 
by  writing  to  the  Defense  Hotline,  The  Pentagon,  Washington,  D.C.  2O301- 
1900.  The  identity  of  each  writer  and  caller  is  fully  protected. 


Acronyms 


ADS 

DFAS 

IPC/CDS 

SNIPS 

SRD-1 

Y2K 


Automated  Disbursing  System 

Defense  Finance  and  Accounting  Service 

Integrated  Paying  and  Collecting/Central  Disbursing  System 

Standard  Negotiable  Items  Processing  System 

Standard  Finance  System  Redesign,  Sub-System  One 

Year  2000 


INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON,  VIRGINIA  22202-2884 


October  12,  1999 


MEMORANDUM  FOR  DIRECTOR,  DEFENSE  FINANCE  AND  ACCOUNTING 

SERVICE 

SUBJECT:  Audit  Report  on  Defense  Disbursing  Year  2000  End-to-End  Testing 
(Report  No.  00-  006) 


We  are  providing  this  report  for  information  and  use.  This  report  is  one  in  a 
series  of  reports  that  the  Inspector  General,  DoD,  is  issuing  in  accordance  with  an 
informal  partnership  with  the  Chief  Information  Officer,  DoD,  to  monitor  the  Defense 
Finance  and  Accounting  Service  efforts  in  addressing  the  year  2000  computing 
challenge.  We  considered  management  comments  on  a  draft  of  this  report  when 
preparing  the  final  report. 

Comments  on  the  draft  of  this  report  conformed  to  die  requirements  of  DoD 
Directive  7650.3  and  left  no  unresolved  issues;  therefore,  no  additional  comments  are 
required. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Ms.  Kimberley  A.  Caprio  at  (703)  604-9139  (DSN  664-9139) 
(kcaprio@dodig.osd.mil),  Mr.  Dennis  M.  Conway  at  (703)  604-9158  (DSN  664-9158) 
(dconway@dodig.osd.mil),  or  Ms.  Jacqueline  J.  Vos  at  (703)  604-9146  (DSN  664- 
9146)  (jvos@dodig.osd.mil).  See  Appendix  E  for  the  report  distribution.  The  audit 
team  members  are  listed  inside  the  back  cover. 

Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 
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(Project  No.  9FG-9029) 

Defense  Disbursing 
Year  2000  End-to-End  Testing 


Executive  Summary 


Introduction.  This  report  is  one  in  a  series  of  reports  that  the  Inspector  General,  DoD, 
is  issuing  in  accordance  with  an  informal  partnership  with  the  Chief  Information 
Officer,  DoD,  to  monitor  DoD  efforts  in  addressing  the  year  2000  computer  challenge. 
For  a  listing  of  audit  projects  addressing  the  issue,  see  the  year  2000  web  page  at 
http://www.ignet.gov. 

The  purpose  of  end-to-end  testing  is  to  verify  that  the  set  of  interrelated  systems 
supporting  an  organizational  function,  such  as  disbursing,  interoperates  as  intended 
during  normal  operating  conditions.  (The  Defense  Finance  and  Accounting  Service 
organized  its  end-to-end  testing  into  seven  “events,”  one  for  each  critical  mission  or 
business  process.  The  disbursing  process  was  desi^ated  as  one  of  the  seven  events.) 
Disbursing  event  testing  was  scheduled  for  completion  by  September  15,  1999. 

Objective.  The  overall  audit  objective  was  to  evaluate  the  effectiveness  of  the  planned 
end-to-end  testing  in  the  DoD  disbursing  process.  Specifically,  we  reviewed  the 
Defense  Finance  and  Accounting  Service  year  2000  end-to-end  master  plan  and  the  four 
disbursing-specific  test  plans  for  conducting  end-to-end  testing. 

Results.  The  Defense  Finance  and  Accounting  Service  Year  2000  Project  Office 
developed  a  sound  overall  methodology  for  conducting  end-to-end  testing.  Also,  the 
disbursing  event  leader  was  aware  of  and  actively  involved  in  planning  for  end-to-end 
testing.  However,  unless  rigorous  testing  of  interfaces  takes  place  between  disbursing 
systems  and  other  systems,  testing  disbursing  as  a  separate  event  may  produce 
inconsistent  or  incomplete  test  results  because  the  disbursing  process  is  an  integral  part 
of  each  of  the  other  six  core  processes.  In  addition,  the  “DFAS  [Defense  Finance  and 
Accounting  Service]  Headquarters  Disbursing  Business  Area  Year  2000  End-to-End 
Test  Plan,”  April  19,  1999  (the  Disbursing  Event  Plan),  was  deficient  in  describing  the 
test  procedures,  processes,  and  resources  that  may  be  required  for  conducting  end-to- 
end  tests.  Also,  the  Disbursing  Event  Plan  did  not  contain  a  master  schedule  of  when 
testing  was  to  begin  or  end,  or  the  criteria  for  successful  completion  of  end-to-end 
testing.  Further,  the  disbursing  event  system  test  plans  lacked  critical  requirements  for 
testing  the  disbursing  event,  procedures  for  developing  test  data  and  expected  test 
results  (a  baseline),  consistency  in  the  use  of  test  scenarios,  and  standardization  in  the 
selection  of  a  test  methodology. 

As  a  result,  the  Disbursing  Event  Plan  and  the  disbursing  system  test  plans  for  end-to- 
end  testing  may  not  hilly  ensure  that  the  processing  and  disbursing  of  payments  will 
continue  unaffected.  Because  some  of  the  issues  identified  in  this  report  might  not  be 
resolved  before  the  completion  of  testing,  DFAS  needed  to  take  additional  risk 
mitigation  measures.  For  details  of  the  audit  results,  see  the  Finding  section  of  the 
report. 


Summary  of  Recommendations.  We  recommend  that  the  Director,  Defense  Finance 
and  Accounting  Service,  initiate  steps  to  include  the  Standard  Negotiable  Items 
Processing  System  in  the  end-to-end  testing  process,  whether  as  part  of  the  disbursing 
event  or  another  event.  We  also  recommend  that  the  Director,  Defense  Finance  and 
Accounting  Service,  initiate  alternative  measures  to  mitigate  the  risks  of  disbursing 
systems  not  being  able  to  successfully  process  data  after  the  year  2000.  Such 
alternative  measures  may  include,  for  example,  performing  supplementary  end-to-end 
tests  of  the  event,  using  code  scanners,  or  expanding  event  contingency  plans. 

Management  Comments.  Management  concurred  with  the  recommendations  in  the 
draft  report  and  took  responsive  action.  The  Director  for  Information  and  Technology, 
Defense  Finance  and  Accounting  Service,  stated  that  the  Standard  Negotiable  Items 
Processing  System  was  included  in  an  end-to-end  test.  The  Defense  Finance  and 
Accounting  Service  will  code  scan  the  disbursing  systems.  The  expected  completion 
date  of  the  scanning  is  October  30,  1999.  See  the  Finding  section  for  a  discussion  of 
management  comments  and  the  Management  Comments  section  for  the  complete  text  of 
the  management  comments. 
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Background 


Addressing  the  Year  2000  Computing  Challenge.  Appendix  A  provides  the 
web  page  for  the  listing  of  audit  projects  that  address  the  year-2000  (Y2K) 
computing  challenge. 

Magnitude  of  Audited  Entity.  The  Defense  Finance  and  Accounting  Service 
(DFAS)  is  the  principal  disbursing  agency  for  DoD  and  is  under  the  direction, 
authority,  and  control  of  the  Under  Secretary  of  Defense  (Comptroller).  DFAS 
is  responsible  for  the  disbursing  offices  that  make  payments  by  U.S.  Treasury 
check,  by  cash,  or  by  electronically  transferring  funds.  At  the  end  of  FY  1998, 
DFAS  was  processing  a  monthly  average  of  9.8  million  payments  to  DoD 
personnel;  1.2  million  commercial  invoices;  600,000  travel  vouchers  or 
settlements;  500,000  savings  bonds;  and  120,000  transportation  bills.  The 
agency’s  monthly  disbursements  totaled  approximately  $24  billion. 

For  Y2K  purposes,  DFAS  identified  45  systems'  as  “mission  critical.”  For 
purposes  of  testing  mission-critical  systems  for  Y2K  compliance,  DFAS 
identified  its  business  processes  considered  critical  and  developed  plans  to  test 
those  business  processes.  Critical  processes  are  defined  as  those  that,  if  not 
performed,  would  preclude  or  immediately  impair  the  disbursal,  pay,  and 
accounting  of  Defense  entitlements.  S^cifically,  DFAS  identified  the  following 
seven  critical  business  processes:  civilian  pay;  military,  retiree,  and  annuitant 
pay;  contract  and  vendor  pay;  transportation  pay;  travel  pay;  accounting;  and 
disbursing. 

End-to-£nd  Testing.  The  “end-to-end”  process  is  the  complete  flow  of  data 
through  a  set  of  interconnected  systems  that  perform  a  core  business  process, 
function,  or  mission.  Data  flow  begins  with  the  initial  input  of  data  to  the  first 
system  and  ends  with  the  final  receipt  of  data  in  the  last  system  and  receipt  of 
output  by  the  user.  The  purpose  of  Y2K  end-to-end  testing  is  to  verify  that  the 
set  of  interrelated  systems  supporting  DFAS  business  processes,  such  as  DoD 
disbursing,  operates  and  appropriately  processes  Y2K-related  data. 

DFAS  Y2K  End-to-End  Master  Plan.  DFAS  issued  the  “DFAS  Y2K  End-to- 
End  Master  Plan,”  version  2.3,  June  21,  1999  (the  DFAS  Master  Plan), 
specifically  for  accomplishing  Y2K-related  end-to-end  testing  of  its  mission- 
critical  business  processes.  DFAS  organized  end-to-end  testing  into  seven 
testing  “events,"  one  for  each  critical  mission  or  business  process.  The  DFAS 
Master  Plan  has  further  divided  each  event  or  business  process  into  “threads.  ” 

A  thread  represents  an  end-to-end  trace  of  data  using  a  minimally  representative 
sample  of  input  data  transformed  through  the  interconnected  set  of  systems  to 
produce  a  minimally  representative  sample  of  ou^ut  data.  Each  event  can 
contain  one  or  more  “threads”  that  track  to  a  critical  business  process. 


‘  We  identified  42  DFAS  mission-critical  systems  in  previous  audit  reports.  DFAS  recently  added  three 
systems  to  its  mission-critical  list:  die  Standard  Accounting  and  Reporting  System  -  One  Bill  Pay,  the 
Standard  Army  Financial  Accounting  and  Reporting  System,  and  the  Standard  Base  Supply  System. 
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The  DFAS  Master  Plan  identifies  roles  and  responsibilities;  assumptions  and 
constraints  related  to  testing;  interfaces  with  non-DFAS  organizations;  and 
requirements  for  planning,  testing,  and  reporting  on  test  results. 

Roles  and  Responsibilities.  DFAS  designated  a  Y2K  project  manager 
and  functional  proponent  at  DFAS  Headquarters  with  overall  Y2K  testing 
responsibility.  DFAS  delegated  that  its  functional  proponents  assign  event 
leaders  and  thread  leaders  to  execute  the  end-to-end  testing.  Also,  DFAS 
assigned  roles  and  responsibilities  to  system  managers  for  controlling  their 
segment  of  die  end-to-end  testing  process. 

Assumptions  and  Constraints.  Because  of  limited  time  and  resources, 
the  DFAS  Master  Plan  acknowledged  constraints  and  identified  assumptions 
related  to  Y2K  end-to-end  testing.  The  assumptions  included,  for  example,  that 
third-party  software  and  computing  platforms  are  Y2K  compliant,  that 
operations  and  compliance  testing  takes  precedence  over  end-to-end  testing,  that 
partner  organizations  will  conduct  their  own  internal  end-to-end  tests  and 
provide  input  for  DFAS,  or  that  all  mission-critical  systems  will  have 
contingency  plans  in  place. 

biterface  Requirements.  The  DFAS  Master  Plan  states  that  each  test 
event  will  include  critical  automated  interfaces  with  other  departments  and 
agencies.  However,  because  of  sizing  limitations  within  the  DoD  megacenters 
that  support  testing,  DFAS  and  the  other  DoD  organizations  may  not  be  able  to 
run  true  end-to-end  tests  simultaneously.  Rather,  each  organization  will 
maintain  sufficient  control  of  its  segment  of  the  end-to-end  testing  process  to 
ensure  the  integrity  of  the  data  flow  from  one  system  to  the  other. 

Planning,  Testing,  and  Reporting.  The  DFAS  Master  Plan  specified 
requirements  for  the  following. 

•  Live  Instead  of  Simulation.  DFAS  planned  to  test  its  business 
processes  under  normal  operating  conditions  when  possible. 
Otherwise,  DFAS  was  to  use  a  “time  machine”  or  simulated 
operating  conditions  and  document  the  reasons  and  associated  risks. ^ 

•  Critical  Dates.  Although  the  DFAS  Master  Plan  did  not  designate 
specific  dates  for  testing,  it  did  recommend  that  testing  cover  the 
following  five  time  periods:  the  FY  2000  crossover,  calendar  year 
2000  crossover,  FY  2001  crossover,  calendar  year  2001  crossover, 
and  leap  year  (February  29,  2000). 

•  Baselmes.  After  testing  those  dates,  DFAS  organizations  were  to 
compare  their  test  results  to  outcomes  previously  captured  as  the 
baseline,  (The  baseline  is  a  set  of  known  end-to-end  test  inputs  and 


*  A  “time  machine"  test  Strategy  involves  setting  system  clocks  to  the  year  2000  and  operating  under 
testing  conditions.  Simulation  is  a  program  that  allows  testers  to  simulate  changing  dates  rather  than 
actually  rtianging  the  dates  during  normal  operations. 
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outputs  captured  from  systems  that  were  certified  as  Y2K  compliant.) 
Each  DFAS  organization  was  to  document  the  discrepancies  between 
each  of  the  tests  and  the  baseline. 

•  Data  Analysis  and  Documentation.  The  DFAS  Master  Plan 
requires  that  each  DFAS  organization  develop  and  document  in  its 
test  plan  a  data  collection  and  analysis  strategy  that  provides 
sufficient  information  to  support  end-to-end  test  design,  results,  and 
analysis. 

Preliminary  Assessment.  On  May  28,  1999,  the  Office  of  the  Inspector 
General,  DoD,  provided  DFAS  with  preliminary  audit  results  in  a  memorandum 
regarding  DFAS  planning  for  end-to-end  testing.  On  June  8,  1999,  the  DFAS 
Director  for  Information  and  Technology  responded  to  the  issues  of  roles  and 
responsibilities,  the  DFAS  Master  Plan  checklists,  interfacing  systems,  critical 
Y2K  crossover  dates,  data  collection  and  analysis,  and  the  transportation  pay 
event.  Office  of  the  Inspector  General,  DoD,  audit  results  and  DFAS  comments 
on  the  disbursing  process  are  summarized  in  the  finding  section  of  this  report. 
Appendix  C  contains  a  copy  of  the  memorandum  that  die  Inspector  General, 
DoD,  issued,  and  Appendix  D  is  the  DFAS  response. 

On  July  14,  1999,  we  met  with  the  DFAS  Y2K  project  officer  to  discuss  our 
concerns  with  testing  of  the  disbursing  process  as  a  separate  event.  Testing  of 
the  disbursing  process  as  a  separate  event  may  produce  inconsistent  or 
incomplete  test  results  because  the  disbursing  process  is  an  integral  part  of  each 
of  the  other  six  events.  If  no  testing  takes  place  on  the  interfaces  between  the 
disbursing  event,  the  other  six  events,  and  systems  external  to  the  disbursing 
event,  the  risk  will  increase  that  the  disbursing  event  may  not  be  sufficiently 
tested  to  prevent  disruptions  at  the  year  2000.  DFAS  had  since  agreed  to 
integrate  the  disbursing  end-to-end  tests  with  all  entitlement  end-to-end  tests. 


Objective 


The  overall  audit  objective  was  to  evaluate  the  effectiveness  of  the  planned  end- 
to-end  testing  of  the  DoD  disbursing  process.  Specifically,  we  reviewed  Ae 
DFAS  Master  Plan  and  the  test  plans  for  conducting  end-to-end  testing  within 
the  disbursing  pay  area.  The  purpose  of  end-to-end  testing  is  to  verify  that  the 
set  of  interrelated  systems  supporting  an  organizational  function,  such  as 
disbursing  pay,  interoperates  as  intended  in  an  operational  environment. 

This  report  addresses  plans  for  conducting  Y2K  end-to-end  testing  within  the 
disbursing  area.  Other  reports  will  address  other  DFAS  functional  areas.  See 
Appendix  A  for  a  discussion  of  the  audit  scope  and  methodology  and  prior  audit 
coverage  related  to  the  audit  objective.  We  will  review  the  implementation  of 
the  end-to-end  test  plans  and  the  adequacy  of  contingency  plans  during  future 
audit  work. 
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Plans  for  End-to-End  Testing  of 
Disbursing  Systems 

The  DFAS  Y2K  Project  Office  developed  a  sound  methodology  for 
conducting  end-to-end  testing.  In  addition,  the  disbursing  event  leader 
was  aware  of  and  actively  involved  in  planning  for  end-to-end  testing. 
However,  DFAS  needed  to  address  the  following  issues: 

•  as  planned,  DFAS  segregated  the  disbursing  event  from  the 
other  six  events  rather  than  including  it  as  the  culmination  of 
each  of  the  odier  events,  and  participation  by  other  events  to 
provide  the  necessary  interfacing  support  and  data  to  the 
disbursing  event  was  slow; 

•  the  “DFAS  Headquarters  Disbursing  Business  Area  Year 
2000  End-to-End  Test  Plan,”  April  19,  1999  (the  Disbursing 
Event  Plan),  focused  on  three  DFAS-owned  systems  that 
perform  disbursing  functions,  but  it  did  not  include  some 
critical  test  requirements  prescribed  by  the  “DoD  Y2K 
Management  Plan,”  Version  2.0,  December  1998,  and  the 
DFAS  Master  Plan.  Specifically,  the  Disbursing  Event  Plan 
was  deficient  in  the  following: 

-  describing  the  test  procedures,  processes,  and  exit 
criteria; 

-  providing  a  master  schedule  of  test  dates;  and 

-  discussing  resources  required  for  Y2K  end-to-end 
testing;  and 

•  the  disbursing  event  system  test  plans  lacked  critical 
requirements  for  completing  tests,  baseline  and  test  data  for 
assessing  test  results,  scenarios  for  testing  data,  and  methods 
for  conducting  tests. 

As  a  result,  DFAS  had  not  yet  minimized  the  risk  of  disruption  to  the 
disbursing  process. 


DoD  Disbursing  Process 

DeHnition  of  Disbursements.  Within  DoD,  disbursements  result  when  DoD 
entitlement  systems  generate  files  that  are  transferred  into  one  of  the  principal 
disbursing  systems.  Entitlement  is  the  legal  right  to  receive  items  of  pay  or 
allowances.  An  entitlement  system  is  one  that  would  calculate  the  amount  that 
would  be  received  by  a  payee,  such  as  an  individual  or  company.  Disbursing 
systems  are  those  systems  that  provide  disbursements  to  the  payee  in  the  amount 
of  Ae  entitlement  either  in  the  form  of  cash,  electronic  funds  transfer,  or  check. 
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Generally,  all  disbursements  funnel  through  DFAS.  Depending  on  the  type  of 
payment,  such  as  cash,  electronic  funds  transfer,  or  check,  the  transaction  is 
forwarded  through  the  Federal  Reserve  Bank  to  a  payee  bank,  or  is  sent  directly 
to  an  individual  or  company. 

Disbursing  files  are  also  routed  directly  to  various  accounting  systems  within 
DoD  to  record  the  transactions  once  completed.  The  following  figure  describes 
the  flow  of  disbursement  transactions  from  entitlement  systems  to  recipients  and 
DoD  accounting  systems. 


IRS  -  Internal  Revenue  Service 
OMB  Office  of  Management  and  Budget 
0PM  -  Office  of  Personnel  Management 
SSA  Social  Security  Administration 


Defense  Disbursing  Process 

DFAS  Process  for  Testing  the  Disbursing  Event.  The  Disbursing  Event  Plan 
required  disbursing  eyent  system  test  plans  to  test  the  following  three  DFAS- 
owned  systems  as  “threads”:  the  Automated  Disbursing  System  (ADS)  at  the 
DFAS  Cleveland  Center;  the  Integrated  Paying  and  Collecting/Central 
Disbursing  System  (IPC/CDS)  at  the  DFAS  Denver  Center;  and  the  Standard 
Finance  System  Redesign,  Sub-System  One  (SRD-1)  at  the  DFAS  centers  in 
Indianapolis,  Indiana;  Columbus,  Ohio;  and  Kansas  City,  Missouri.  Testing  of 
the  disbursing  function  began  on  June  4,  1999,  and  the  last  testing  is  scheduled 
on  October  30,  1999. 
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Additional  Disbursing  System.  Although  DFAS  identified  three  threads  for 
end-to-end  testing  in  the  Disbursing  Event  Plan,  the  DoD  Biennial  Financial 
Management  Improvement  Plan  (undated),  Appendix  B,  “DoD  FY  1998 
Systems  Inventory,”  also  included  a  fourth  disbursing  system— the  Standard 
Negotiable  Items  Processing  System  (SNIPS).  DFAS  did  not  include  SNIPS  in 
its  Disbursing  Event  Plan  because  the  disbursing  event  leader  could  not  confirm 
that  it  was  a  DFAS-owned  system.  However,  the  disbursing  event  leader  later 
stated  that  SNIPS  should  be  considered  a  disbursing  system  for  the  event. 

Because  SNIPS  was  included  as  a  mission-critical  disbursing  system  in  the 
Biennial  Plan  but  was  not  included  in  the  Disbursing  Event  Plm,  the  disbursing 
event  leader  should  include  it  in  the  end-to-end  testing  of  the  disbursing  event. 
DFAS  needs  to  ensure  that  SNIPS  is  included  in  the  testing,  whether  as  part  of 
the  disbursing  event  or  through  alternative  means  to  mitigate  the  risk  of  possible 
disbursing  disruptions. 

Systems  Interfacing  With  DFAS  Disbursing  Systems.  The  four  DFAS-owned 
disbursing  systems  interface  with  27  mission-critical  systems  and  38  non- 
mission-critical  systems.  Of  the  65  interfacing  systems,  41  systems  feed  data 
into  the  four  DFAS  disbursing  systems,  and  24  systems  receive  data  from  the 
DFAS  systems,  to  include  the  Federal  Reserve  Bank  systems.  For  details  on  the 
four  DFAS  disbursing  systems,  see  Appendix  B. 


Disbursing  as  a  Separate  Event 

DFAS  Designation  of  Disbursing  as  a  Separate  Event.  For  Y2K  pu^oses, 
DFAS  identified  the  disbursing  process  as  one  of  its  seven  mission-critical 
processes,  or  events.  DFAS  did  not  include  the  disbursement  process  as  an 
integral  part  of  entitlement  events  such  as  military  pay  or  civilian  pay.  DFAS 
chose  to  separate  the  disbursing  event  from  the  other  events  because  disbursing 
is  normally  considered  a  separate  business  process  within  DFAS.  However, 
DFAS  disbursing  operations  do  not  include  the  processing  of  entitlement  data; 
rather,  disbursing  systems  rely  on  data  from  other  business  processes,  such  as 
military  pay,  to  provide  data  for  testing  of  the  disbursing  event.  As  such, 
testing  of  the  disbursing  event  relies  on  interfaces  and  coordination  with  the 
other  six  events. 

Coordination  With  Interfacing  Organizations.  The  Disbursing  Event  Plan 
states  that  scheduling  for  testing  is  100  percent  dependent  upon  the  coordination 
of  system  interfaces  within  the  DFAS  network  as  well  as  organizations  such  as 
the  Federal  Reserve  Bank,  Defense  megacenters,  and  financial  institutions.  The 
Disbursing  Event  Plan  contained  a  list  of  interfacing  systems  for  each  of  the 
disbursing  systems.  However,  the  disbursing  event  leader  was  unable  to 
determine  the  systems  or  interfaces  for  each  event  leader  to  test  because  the 
disbursing  event  leader  did  not  have  the  authority  to  dictate  to  the  other  six 
event  leaders  or  system  managers  when  testing  should  be  started  or  ended,  or 
the  systems  that  would  be  tested.  In  addition,  DFAS  personnel  stated  that  end- 
to-end  testing  of  disbursing  systems  was  dependent  on  the  dates  of  testing 
established  with  the  Federal  Reserve  Bank  FEDLINE  system.  The  disbursing 
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event  leader  took  the  initiative  to  coordinate  an  end-to-end  test  schedule  with 
Federal  Reserve  Bank  personnel,  but  was  limited  to  testing  with  the  Federal 
Reserve  Bank  on  specific  dates  and  with  specific  test  data.  The  disbursing  event 
leader  published  the  Federal  Reserve  Bank  testing  dates  for  the  other  six  event 
leaders  to  use  in  selecting  and  coordinating  test  dates.  As  of  June  28,  1999,  the 
disbursing  event  leader  stated  that  only  the  military  pay  and  civilian  pay  event 
leaders  had  coordinated  testing  dates.  However,  on  August  10,  1999,  the 
disbursing  event  leader  at  DFAS  Center  Kansas  City,  Missouri,  indicated  that 
coordination  with  the  other  six  events  was  improving. 

Testing  of  the  disbursing  process  as  a  separate  event  may  produce  inconsistent 
or  incomplete  test  results  because  the  disbursing  process  is  an  integral  part  of 
each  of  the  other  six  events.  Therefore,  thorough  testing  of  interfaces  between 
the  disbursing  event,  the  other  six  events,  and  systems  external  to  the  disbursing 
event  is  especially  crucial. 

The  DFAS  Y2K  project  manager  should  obtain  agreements  for  end-to-end 
testing  of  systems  interfacing  with  the  four  disbursing  systems.  Not  much  time 
remains  for  scheduling  of  tests  with  the  other  six  events.  Any  problems  in 
completing  this  coordination  should  be  reported  immediately  to  senior 
management. 


DFAS  Headquarters  Disbursing  Event  Planning 


The  Disbursing  Event  Plan  did  not  include  some  critical  test  requirements 
prescribed  by  die  DoD  Y2K  Management  Plan  and  the  DFAS  Master  Plan,  The 
DoD  Y2K  Management  Plan  states  that  an  event  plan  should  identify  the 
mission-critical  systems  to  be  tested  and  the  expected  results.  The  DFAS 
Master  Plan  states  that  each  event  plan  should  contain  descriptions  of  test 
procedures,  processes,  exit  criteria,  schedules,  and  the  resources  required  for 
conducting  the  test. 

The  Disbursing  Event  Plan  did  identify  the  mission-critical  systems  to  be  tested 
for  three  of  the  four  disbursing  systems,  but  it  did  not  provide  a  description  of 
expected  results.  Also,  the  Disbursing  Event  Plan  was  deficient  in  describing 
the  procedures,  processes,  and  exit  criteria.  In  addition,  the  Disbursing  Event 
Plan  did  not  provide  a  master  schedule  of  when  testing  was  to  begin  or  end,  nor 
did  it  provide  a  discussion  on  the  types  of  resources  that  may  be  required  in 
conducting  the  tests.  Further,  the  Disbursing  Event  Plan  did  not  include  the 
necessary  details  for  testing  the  entitlement  and  accounting  systems  that  provide 
data  to  the  disbursing  systems. 

The  Disbursing  Event  Plan  indicated  that  the  system  managers  at  each  of  the 
DFAS  centers  were  responsible  for  developing  the  individual  test  plans  for  those 
disbursing  systems  that  operate  at  the  particular  centers.  The  Disbursing  Event 
Plan  also  states  that  test  setup,  execution,  recovery  plans,  criteria  for  test 
readiness,  test  data,  use  of  checklists,  data  collection  and  analysis,  and  test  exit 
criteria  would  be  described  in  each  DFAS  center  test  plan  in  which  disbursing 
systems  operate.  The  system  test  plans  that  the  system  managers  developed  did 
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not  include  critical  requirements  for  testing,  collection  of  baseline  data  for 
comparison  to  actual  test  results,  consistent  use  of  DFAS  Headquarters- 
developed  test  scenarios,  or  a  determination  of  a  specific  test  methodology  for 
using  “time  machines.” 

By  allowing  each  DFAS  center  to  develop  a  disbursing  end-to-end  test  plan 
separately,  a  higher  risk  exists  that  test  plans  may  not  achieve  the  requirements 
described  in  the  DFAS  Master  Plan.  The  DFAS  disbursing  event  leader  needs 
to  t^e  a  more  proactive  role  in  ensuring  that  test  setup,  execution,  recovery, 
and  exit  criteria  are  consistently  applied  throughout  the  testing  of  the  disbursing 
event. 


Disbursing  Event  System  Test  Plans 

DFAS  system  managers  independently  developed  a  disbursing  event  system  test 
plan  for  the  end-to-end  testing  of  each  of  the  four  DFAS-owned  disbursing 
systems.  However,  the  disbursing  event  system  test  plans  need  additional  detail 
to  adequately  address  the  requirements  described  in  die  Disbursing  Event  Plan. 
Specifically,  the  individual  disbursing  event  system  (thread)  test  plans  lacked  the 
following: 


•  critical  requirements  for  testing, 

•  requirements  to  collect  baseline  and  test  data, 

•  consistent  use  of  test  scenarios,  and 

•  determination  of  a  testing  methodology. 

Critical  Requirements  for  Testing.  The  disbursing  event  system  test  plans 
lacked  critical  requirements  for  testing.  For  example,  the  Disbursing  Event 
Plan  states  that  the  system  test  plans  should  provide  details  on  test  setup, 
execution,  recovery  plans,  criteria  for  test  readiness,  test  data,  use  of  checklists, 
data  collection  and  analysis,  and  test  exit  criteria.  However,  the  four  disbursing 
test  plans  included  only  minimal  details  on  test  setup,  execution,  and  recovery. 
The  four  test  plans  provided  no  detail  describing  the  predicted  results.  In 
addition,  the  four  test  plans  lacked  a  master  schedule  for  testing  the  various 
interfaces  with  the  mission-critical  systems,  although  each  of  the  plans  did 
provide  a  start  and  ending  date.  Further,  the  four  test  plans  did  not  always 
indicate  the  files  or  interfaces  that  were  to  be  tested  and  when  specific  interfaces 
were  to  be  tested. 

Although  each  of  the  plans  discussed  resources,  the  test  plans  provided 
insufficient  information  on  the  types  of  resources  that  may  be  required  in 
conducting  the  end-to-end  tests.  For  example,  the  IPC/CDS  test  plan  indicated 
the  need  for  a  computer  and  a  printer,  but  it  did  not  discuss  the  need  for 
additional  resources  such  as  heating  and  air  conditioning,  local  area  networks, 
testers,  or  space  requirements.  Also,  the  IPC/CDS  test  plan  assumed  that 
testing  platforms  would  be  available  when  needed.  Although  the  SRD-1  test 
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plan  specifically  addressed  the  requirement  for  resources  such  as  administrative 
supplies,  telephone  lines,  and  arrangements  with  direct  and  indirect  computer 
support  personnel  for  extended  work  hours  to  support  the  test,  the  plan  did  not 
state  whether  the  resources  were  committed  for  testing. 

Also,  the  end-to-end  test  plans  were  inconsistent  in  their  approach  for  testing  the 
disbursing  event.  For  example,  the  end-to-end  test  plans  for  ADS  included  the 
requirement  to  establish  an  inter-organizational  end-to-end  test  team.  The  team 
was  responsible  for  developing  test  scenarios  and  exit  criteria.  Conversely,  the 
DFAS  Denver  Center  was  procuring  assistance  for  developing  the  IPC/CDS  test 
plan  from  the  Standard  Systems  Group  at  Maxwell  Air  Force  Base.  Further,  the 
Disbursing  Event  Plan  did  not  identify  specific  disbursing  processes  that  would 
be  tested. 

The  lack  of  detailed  guidance  in  the  Disbursing  Event  Plan  contributed  to  the 
inconsistent  development  of  the  system  test  plans.  Without  standard  guidelines 
for  developing  detailed,  consistent  test  plans,  a  higher  risk  exists  for  inconsistent 
testing  for  the  four  DFAS-owned  disbursing  systems.  Because  test  setup, 
execution,  and  recovery  are  critical  to  the  success  of  testing,  the  disbursing 
event  leader  should  assist  the  systems  managers  of  the  four  DFAS-owned 
disbursing  systems  in  identifying  critical  elements  and  resources  to  ensure  a 
successfol  test  so  that  disbursing  disruptions  are  kept  to  a  minimum. 

Collection  of  Baseline  and  Test  Data.  The  DFAS  Headquarters  did  not 
require  the  collection  of  data  for  establishing  a  baseline  to  compare  with  test 
results.  Also,  DFAS  did  not  establish  a  standard  method  for  collecting  the  data. 
The  plans  for  testing  each  disbursing  system  provided  limited  details  on  the 
methodology  to  collect  data  for  the  end-to-end  tests. 

Establishing  a  Baseline.  The  Disbursing  Event  Plan  states  that  the 
methodology  for  conducting  tests  should  be  presented  in  each  DFAS  center’s 
end-to-end  test  plan.  However,  none  of  the  test  plans  for  any  of  four  DFAS 
disbursing  threads  included  the  requirement  to  obtain  data  to  establish  a  baseline 
to  compare  against  actual  test  results.  DFAS  management  should  encourage  the 
capturing  of  baseline  information  before  beginning  testing  so  that  results  can  be 
compared  and  discrepancies  corrected  before  the  year  2000.  By  collecting  data 
for  establishing  a  baseline,  DFAS  could  better  ensure  that  all  disbursing 
processes  were  tested  properly,  and  that  they  meet  Y2K  requirements. 

Collecting  Data  for  Tests.  The  DoD  Y2K  Management  Plan  states 
that  test  plans  should  specify  in  detail  the  Y2K  data  that  need  to  be  collected, 
those  responsible  for  analyzing  the  data,  the  methods  for  analyzing  the  data,  the 
facilities  to  be  used  to  analyze  the  test  results,  the  mechpism  for  sending  the 
data  to  the  analysis  site,  the  necessary  support,  and  the  identification  of  analysis 
resources.  The  DFAS  Master  Plan  requires  each  DFAS  organization  to  develop 
a  strategy  for  collecting  and  analyzing  data.  The  strategy  should  assist  wiA 
designing  end-to-end  tests,  capturing  test  results,  and  performing  post-testing 
actions.  Establishing  a  standard  format  for  collecting  test  data  would  also 
provide  for  consistent  data  collection. 
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The  plans  for  testing  each  disbursing  system  provided  limited  details  on  how 
data  would  be  collected  for  the  end-to-end  tests.  For  example,  the  SRD-1  test 
plan  simply  stated  that  upon  completion  of  each  test,  the  essential  elements  for 
analysis  would  be  signed  by  a  representative  witnessing  the  test.  In  addition, 
although  the  test  plan  did  include  a  form  for  recording  test  results  and  obtaining 
signatures  from  test  participants,  the  plan  did  not  specify  the  information  to  be 
collected  or  reported.  Further,  the  IPC/CDS,  SNIPS,  and  ADS  test  plans  did 
not  mention  how  or  what  data  would  be  collected  or  reported. 

Although  the  collection  of  baseline  data  was  not  included  in  the  system  test 
plans,  baseline  data  collected  during  testing  would  alleviate  testers’  difficulty  in 
determining  whether  the  actual  results  received  during  the  end-to-end  tests  are 
reasonable.  By  not  having  baseline  data  defined  in  advance,  DFAS  increases 
the  risk  that  the  disbursing  end-to-end  tests  would  not  be  adequately  performed 
or  tbat  the  results  would  not  be  accurate.  Without  establishing  standard  test 
collection  formats,  the  data  analysis  may  not  be  consistent  or  accurate. 

Consistent  Use  of  Test  Scenarios.  The  system  test  plans  for  the  four 
disbursing  threads  did  not  include  all  of  the  test  scenarios  reflected  in  the 
Disbursing  Event  Plan.  The  Disbursing  Event  Plan  showed  three  separate  test 
scenarios  for  each  of  the  four  threads.  Those  three  scenarios  were  as  follows. 

Test  Scenario  1.  Test  Scenario  1  would  require  the  duplication  of  a 
prior  pay  file  (such  as  civilian  pay)  that  would  be  processed  through  the 
disbursing  systems  to  the  Federal  Reserve  Bank.  The  dates  in  the  files  would  be 
manipulated  to  future  dates.  The  Federal  Reserve  Bank  would  receive,  test,  and 
make  distribution  of  the  files  to  financial  institutions.  (The  files  would  include 
some  bad  records;  the  financial  institutions  were  to  return  those  records  to  the 
disbursing  system.) 

Test  Scenario  2.  In  Test  Scenario  2,  entitlement  systems  would  send  a 
file  to  the  disbursing  system  with  manipulated  future  dates.  From  the  disbursing 
system,  the  file  would  be  processed  to  the  Federal  Reserve  Bank. 

Test  Scenario  3.  In  Test  Scenario  3,  users  and  custoiners  would  input 
data  to  entitlement  systems  that  would  process  transactions,  with  Y2K  test  dates, 
through  the  disbursing  systems.  The  disbursing  systems  would  process  the 
transactions  to  the  Federal  Reserve  Bank  for  distribution  to  the  applicable 
financial  institutions.  Transactions  not  accepted  by  the  financial  institutions 
were  to  be  returned  to  disbursing. 

None  of  the  four  test  plans  specifically  stated  that  testing  would  include  the  three 
test  scenarios.  Three  of  the  four  test  plans  provided  onfy  brief  descriptions  of 
testing.  For  example,  the  IPC/CDS  test  plan  stated  that  it  would  have  tiered 
testing  to  include  entitlement  and  collection  system  interfaces  and  that  it  would 
process  electronic  funds  transfer  and  check  payment  files  to  the  Federal  Reserve 
Bank.  Also,  the  plan  stated  that  the  system  would  process  a  disbursing  officer’s 
daily,  monthly,  and  annual  accountability  reports. 

The  SRD-1  test  plan  stated  that  DFAS  personnel  would  generate  data  input  at 
Ae  test  site  or  data  would  be  available  from  the  batch  processing  of  external 


10 


systems.  Although  the  ADS  test  plan  listed  tasks  to  be  completed,  it  did  not 
provide  how,  when,  where,  or  who  would  complete  the  tasks.  Only  the  SNIPS 
test  plan  provided  a  very  limited  description  of  test  scenarios,  but  even  the 
SNIPS  plan  did  not  describe  what  the  scenarios  would  accomplish,  or  the 
expected  test  results. 

The  four  disbursing  system  test  plans  did  not  ensure  that  testing  would  be 
completed  with  the  Federal  Reserve  Bank  or  with  the  entitlement  systems. 
Essentially,  the  plans  were  very  limited  in  their  description  of  what  was  to  be 
tested  and  when,  how,  or  who  would  conduct  the  tests.  The  three  test  scenarios 
described  in  the  Disbursing  Event  Plan  provide  for  a  complete  assessment  of  the 
disbursing  process.  DFAS  needs  to  use  the  three  test  scenarios  to  ensure 
consistent  testing  of  its  disbursing  systems,  to  ensure  that  test  results  are 
reliable,  and  to  ensure  that  the  four  disbursing  systems  will  work  as  intended 
when  the  year  2000  occurs. 

Determination  of  Testing  Methodology.  The  testing  methodology  presented  in 
the  system  test  plans  did  not  ensure  that  end-to-end  testing  would  occur  under 
conditions  similar  to  normal  operating  conditions.  The  DFAS  Master  Plan 
required  tests  to  be  performed  during  normal  operations,  if  possible,  and  to  use 
“time  machines”  when  normal  operations  were  not  feasible.  A  “time  machine” 
changes  the  test  environment  (computer  software  and  hardware)  to  the 
year  2000  and  uses  software  and  computer  equipment  representative  of  normal 
operations. 

None  of  the  four  disbursing  system  test  plans  indicated  that  tests  would  be 
conducted  under  normal  operating  conditions.  Only  the  IPC/CDS  system  test 
plan  indicated  that  it  would  use  the  “time  machine”  methodology.  The  other 
three  disbursing  test  plans  did  not  indicate  a  testing  methodology. 

The  DFAS  Master  Plan  stated  that  if  testing  under  normal  operating  conditions 
or  the  use  of  “time  machine”  testing  was  not  possible,  the  testing  organization 
should  use  simulation  software,  document  the  reasons  for  usinjg  the  software, 
and  describe  Ae  associated  risk.  However,  use  of  the  simulation  methodology 
would  cause  DFAS  to  incur  a  higher  degree  of  risk  of  potential  Y2K  failure 
because  the  simulation  software  bypasses  computer  codes  that  might  perform 
changes  to  dates. 

In  the  absence  of  testing  under  normal  operating  conditions,  DFAS  must 
encourage  its  system  managers  to  use  the  “time-machine”  methodology  when 
testing  end-to-end.  Use  of  simulation  testing  or  any  other  testing  methodology 
may  not  provide  adequate  assurance  that  the  software  and  related  computer 
equipment  will  operate  as  intended  when  the  year  2000  occurs. 

Alternative  Measures.  Senior  DoD  managers  have  emphasized  that  systems 
should  undergo  independent  validation  of  the  testing  or  testing  results.  Where 
testing  has  been  inadequate  or  in  need  of  retesting,  alternative  measures  need  to 
be  considered  given  time  constraints.  Alternative  measures  may  include,  for 
example,  retesting  or  the  use  of  automated  tools.  The  use  of  automated  Y2K 
tools  called  code  scanners  has  identified  hundreds  of  Y2K  errors  in  systems 
previously  certified  as  compliant.  Code  scanners  have  been  made  available  free 
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of  charge  within  DoD  and  may  be  used  at  any  time,  during  or  after  the  testing 
process.  In  addition,  it  may  be  appropriate  to  alert  those  responsible  for 
contingency  planning  to  expand  plans  as  necessary. 


Actions  Taken  by  DFAS 


Distribution  of  Revisions  to  the  DFAS  Master  Plan.  DFAS  initially  did  not 
address  how  its  revised  Master  Plan  would  be  distributed  to  disbursing 
personnel  planning  for  end-to-end  tests.  However,  on  May  18,  1999,  DFAS 
stated  that  revisions  to  its  Master  Plan  would  be  posted  on  a  web  site.  Further, 
DFAS  stated  that  any  major  changes  to  the  DFAS  Master  Plan  would  be 
forwarded  to  the  event  leaders  for  implementation  and  further  dissemination. 
Those  actions  would  allow  the  personnel  to  know  about  changes  in  requirements 
and  to  make  necessary  adjustments. 

Interim  Inspector  General,  DoD,  Results  of  Audit.  On  May  28,  1999,  the 
Inspector  General,  DoD,  issued  a  memorandum  to  the  Director  for  Information 
and  Technology,  DFAS,  on  six  issues  concerning  the  DFAS  end-to-end  test 
plans  (see  Appendix  C).  The  Director  for  Information  and  Technology 
responded  in  a  memorandum  on  June  8,  1999  (see  Appendix  D).  Three  of  the 
six  issues  apply  to  the  disbursing  event,  which  were  the  master  plan  checklists, 
interfacing  systems,  and  data  collection  and  analysis. 

Master  Plan  Checklists.  Our  memorandum  indicated  a  concern  that  the 
checklists  in  the  DFAS  Master  Plan  were  not  mandatory.  DFAS  responded 
that,  although  it  would  encourage  the  use  of  the  checklists,  it  would  not  mandate 
the  completion  of  the  checklists. 

Interfacing  Systems.  DFAS  relied  heavily  on  interfacing  systems  to 
provide  the  majority  of  data  included  in  its  systenis.  Our  memorandum 
emphasized  the  need  for  coordination  and  compatibility  of  data  exchanged  with 
interfacing  systems  for  successful  Y2K  end-to-end  tests.  DFAS  agreed  that 
coordination  and  compatibility  is  essential  and  stated  that  it  established 
agreements  with  all  of  its  interfacing  partners.  However,  the  test  plans  did  not 
provide  details  on  milestones  and  schedules  to  ensure  that  coordination  with 
interfacing  partners  was  sufficient. 

Data  Collection  and  Analysis.  In  response  to  our  memorandum,  DFAS 
stated  that  it  would  define  specific  exit  criteria  for  the  test  planning  phase.  In 
addition,  DFAS  hired  the  Joint  Interoperability  Test  Command  to  validate  and 
verify  the  end-to-end  planning  and  testing  efforts.  DFAS  believes  that  the  Joint 
Interoperability  Test  Command  will  analjrze  and  document  specific  risks 
associated  with  data  collection  and  analysis  procedures,  in  sufficient  time  for 
DFAS  to  take  appropriate  action.  DFAS  also  stated  that  it  would  meet  with 
thread  participants  and  emphasize  the  need  for  adequate  documentation  of  data 
collection  and  analysis  procedures. 
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Summary 


DFAS  managers  developed  a  sound  overall  methodology  for  conducting  end-to- 
end  testing.  Also,  the  managers  were  aware  of  and  actively  involved  in 
planning  for  end-to-end  testing.  However,  because  testing  of  disbursing  was 
performed  as  a  separate  event,  the  results  may  be  inconsistent  or  incomplete 
because  the  disbursing  process  is  reliant  on  input  from  each  of  the  other  six 
events.  If  interfaces  do  not  have  proper  testing  between  the  disbursing  event, 
the  other  six  events,  and  systems  external  to  the  disbursing  event,  the  risk  will 
increase  that  the  disbursing  event  may  not  be  sufficiently  tested  to  prevent 
disruptions  at  the  year  2000. 

In  addition,  the  Disbursing  Event  Plan  was  deficient  in  describing  the  test 
procedures,  processes,  and  resources  that  may  be  required  for  conducting  end- 
to-end  tests.  Also,  the  event  plan  did  not  contain  a  master  schedule  of  when 
testing  was  to  begin  or  end,  nor  did  it  contain  the  criteria  for  successful 
completion  of  end-to-end  testing.  Further,  the  disbursing  system  test  plans 
lacked  critical  requirements  for  testing  the  disbursing  event,  procedures  for 
developing  test  data  and  expected  test  results  (a  baseline),  consistency  in  the  use 
of  test  scenarios,  and  standardization  in  the  selection  of  a  test  methodology.  As 
a  result,  the  disbursing  event  plan  and  the  disbursing  system  tests  for  end-to-end 
testing  may  not  fhlly  ensure  that  the  processing  and  disbursing  of  payments  will 
continue  unaffected.  Because  some  of  the  issues  identified  in  this  report  might 
not  be  resolved  before  the  completion  of  testing,  DFAS  needed  to  initiate 
alternative  measures  to  mitigate  the  risks  of  disbursing  systems  not  being  able  to 
successfully  process  data  after  the  year  2000.  Alternative  measures  may 
include,  for  example,  performing  supplementary  end-to-end  tests  of  the  event, 
using  code  scanners,  or  expanding  event  contingency  plans. 

Recommendations  and  Management  Comments 


We  recommend  that  the  Director,  Defense  Finance  and  Accounting  Service: 

1.  Initiate  steps  to  include  the  Standard  Negotiable  Items 
Processing  System  in  the  end-to-end  testing  process,  whether  as  part  of  the 
disbursing  event  or  another  event. 

2.  Initiate  alternative  measures  to  mitigate  the  risks  of  disbursing 
systems  not  being  able  to  successfully  process  data  after  the  year  2000. 

Such  alternative  measures  may  include  perfonning  supplementary  end-to- 
end  tests  of  the  event,  using  code  scanners,  or  expanding  event  contingency 
plans. 

Management  Comments.  The  Director  for  Information  and  Technology, 
DFAS,  concurred,  stating  that  the  Standard  Negotiable  Items  Processing  System 
was  included  in  an  end-to-end  test.  The  DFAS  will  code  scan  the  disbursing 
systems.  The  expected  completion  date  of  the  scanning  is  October  30,  1999. 
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Appendix  A.  Audit  Process 


This  report  is  one  in  a  series  of  reports  being  issued  by  the  Inspector  General, 
DoD,  in  accordance  with  an  informal  partnership  with  the  Chief  Information 
Officer,  DoD,  to  monitor  DoD  efforts  to  address  the  Y2K  computing  challenge. 
For  a  list  of  audit  projects  addressing  the  issue,  see  the  Y2K  web  page  on  IGnet 
at  http://www.ignet.gov. 

Scope  and  Methodology 


Work  Performed.  We  performed  audit  work  within  the  Defense  Finance  and 
Accounting  Service.  Also,  we  reviewed  the  DFAS  Master  Plan  and  its  event 
plans,  and  we  held  discussions  with  DFAS  managers  to  obtain  additional 
information  and  clarification  on  roles  and  responsibilities  of  its  Y2K  managers. 

We  reviewed  Y2K  reporting  requirements  and  policies  issued  by  the  Office  of 
the  Secretary  of  Defense  and  DFAS.  More  specifically,  we  reviewed  the  “DoD 
Year  2000  Management  Plan,”  Version  2.0,  December  1998;  the  “DFAS  Y2K 
End-to-End  Master  Plan,”  Version  2.3,  June  21,  1999;  the  “DFAS  Year  2000 
Management  Plan,”  Version  2.1,  June  1999;  the  DFAS  Regulation  8000. 1-R, 
“Information  Management  Policy  and  Instructional  Guidance,”  Appendix  E, 
Chapter  3,  “Test  and  Evaluation,”  October  15,  1998;  and  the  DoD  Financial 
Management  Regulation,  Volume  5,  “Disbursing,”  August  1998. 

In  addition,  we  reviewed  the  test  plan  documentation  for  the  four  principal  DoD 
disbursing  systems— the  Standard  Finance  System  Redesign,  Sub-System  One; 
the  Integrated  Paying  and  Collecting/Central  Disbursing  System;  the  Automated 
Disbursing  System;  and  the  Standard  Negotiable  Items  Processing  System. 

DoD- Wide  Corporate-Level  Government  Performance  and  Results  Act 
Goals.  In  response  to  the  Government  Performance  and  Results  Act,  the 
Department  of  Defense  has  established  two  DoD-wide  goals  and 
seven  subordinate  performance  goals.  This  report  pertains  to  achievement  of 
the  following  goal  and  subordinate  performance  goal: 

Goal  2;  Prepare  now  for  an  uncertain  future  by  pursuing  a  focused 
modernization  effort  that  maintains  U.S.  qualitative  superiority  in  key 
warfighting  capabilities.  Transform  the  force  by  exploiting  the  Revolution  in 
Military  Affairs,  and  reengineer  the  Department  to  achieve  a  21st  century 
infrastructure.  Performance  Goal  2.2:  Transform  U.S.  military  forces  for  the 
fiiture.  (OO-DoD-2.2) 

General  Accounting  Office  Hig^-Risk  Area.  The  General  Actuating  Office 
has  identified  several  high-risk  areas  in  DoD.  This  report  provides  coverage  of 
the  Information  Management  and  Technology  high-risk  area.  In  its 
identification  of  risk  areas,  the  Y2K  problem  has  been  specifically  designated  as 
high  risk  by  the  General  Accounting  Office. 
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Use  of  Computer-Processed  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Audit  Type,  Dates,  and  Standards.  We  performed  this  financial-related  audit 
by  reviewing  event  and  test  plans  dated  from  April  through  June  1999,  in 
accordance  with  auditing  standards  issued  by  the  Comptroller  General  of  the 
United  States,  as  implemented  by  the  Inspector  General,  DoD. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  DoD.  Further  details  are  available  on  request. 

Management  Control  Program.  We  did  not  review  the  managenient  control 
program  related  to  the  overall  audit  objective  because  DoD  recognized  the  Y2K 
issue  as  a  material  management  control  weakness  area  in  the  FY  1998  Annual 
Statement  of  Assurance. 


Summary  of  Prior  Coverage 

The  General  Accounting  Office  and  the  Inspector  General,  DoD,  have 
conducted  multiple  reviews  related  to  Y2K  issues.  General  Accounting  Office 
reports  can  be  accessed  over  the  Internet  at  http://www.gao.gov.  Inspector 
General,  DoD,  reports  can  be  accessed  at  http://www.dodig.osd.mil. 
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Appendix  B.  DoD  Disbursing  Systems 

Standard  Finance  System  Redesign,  Sub-System  One  (SRD-1).  SRD-1 
makes  disbursements  at  the  installation  and  DFAS-center  levels  of  operations. 
The  system  includes  all  disbursement  functions  and  the  reporting  requirements 
within  the  finance  and  accounting  offices. 

Standard  Negotiable  Items  Processing  System  (SIWS).  SNIPS  produces 
payroll  checks  for  the  active  and  reserve  Army  military  units.  Payroll  checks 
are  processed  through  the  electronic  funds  transfer  system. 

Integrated  Paying  and  Collecting/Central  Disbursing  System  (IPC/CDS). 
The  IPC/CDS  is  composed  of  approximately  149  programs  that  create  check 
and  electronic  funds  transfer  payments,  process  cashier  transactions,  and 
prepare  accountability  reports  in  both  U.S.  dollars  and  foreign  currency.  The 
IPC/CDS  system  operates  at  the  DFAS  Denver  Center;  however,  program 
management  responsibility  for  the  systems  was  transferred  from  the  DFAS 
Denver  Center  to  the  Standard  Systems  Group  at  Maxwell  Air  Force  Base, 
Montgomery,  Alabama,  in  1991.  CDS  will  replace  IPC  and  is  expected  to  be 
deployed  in  October  1999. 

Automated  Disbursing  System  (ADS).  ADS  provides  automated  disbursing 
and  accounting  pay  processing  in  the  DFAS  Cleveland  Center  and  eliminates  the 
need  for  manual  posting  of  daily  transactions  to  a  cashbook  or  spreadsheet. 

ADS  is  responsible  for  providing  support  to  approximately  18  interfaces.  Also, 
ADS  processes  payment  and  collection  files  submitted  from  entitlement  systems, 
produces  accountability  reports  for  the  disbursing  officer,  and  passes  payment 
information  to  the  Federal  Reserve  Bank  for  electronic  funds  transfer  payments. 
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Appendix  C.  Inspector  General,  DoD,  Interim 

Report 


INSPECTOR  GENERAL 
OePARTMENT  OF  DEFENSE 
400  ARMV  NAVY  DRIVE 
ARLINQTON.  VIRGINIA  22202-2884 


mm  2  B  009 


MEMORANDUM  FOR  DIRECTOR  FOR  INFORMATION  AND  TECHNOLOGY, 
DEFENSE  FINANCE  AND  ACCOUNTING  SERVICE 

SUBJECT:  Status  of  Audits  of  Finance  Functional  Area  Year  2000  End-io-End  Tests 

In  April  1999,  we  initiated  the  following  audit  projects  to  evaluate  the 
effectiveness  of  Y2K  end-to-end  testing  by  the  Defense  Finance  and  Accounting 
Services  (DFAS).  The  projects  were  aligned  by  ftinctional  area  in  accordance  with  the 
functional  break-out  identified  by  DFAS 


Civilian  Pay 

Military /Rctiree/Annuitant  Pay 

Vendor/Contractor  Pay 

Transportation  Pay 

Disbursing 

Accounting 

Travel  Pay 


Project  9FG-9025 
Project  9FG-9026 
Project  9FG-9027 
Project  9FG-9028 
Project  9FG-9029 
Project  9FG-9030 
Project  9FG-9031 


Our  review,  to  date,  has  focused  on  analyzing  the  adequacy  of  test  plans  for  the 
seven  areas.  Wc  evaluated  the  adequacy  of  the  DFAS  Y2K  test  plans  using 
requirements  contained  in  the  DoD  Y2K  Management  Plan,  Version  2.1,  Appendix  I; 
the  DFAS  Y2K  Management  Plan,  Version  1 .0;  the  DFAS  Y2K  End^to-End  Master 
Plan,  Version  2.1;  the  DFAS  Regulation  8000.1-R,  “Infonnation  Management  and 
Instruction  Guidance,**  Version  5  0;  and  the  GAO  Operational  Evaluation  Assc^ment 
Tool.  We  anticipate  future  audits  will  assess  test  results  and  contingency  planning 
efforts  by  DFAS 

Because  of  the  urgency  of  Y^  2000  efforts,  our  intent  is  to  communicate 
potential  areas  of  concern  as  quickly  as  possible  so  that  management  may  address  these 
issues  in  a  timely  manner.  The  attachment  to  this  memorandum  reports  the  initial 
results  of  our  review.  During  our  preliminary  review,  we  identified  concerns  regarding 
the  adequacy  of  DFAS  planning  efforts  for  functional  end-to-end  testing.  If  these 
concerns  aic  not  addressed,  there  is  increased  risk  that  DFAS  end-to-end  testing  may 
not  detect  a  significant  Y2K  problem.  We  may  include  these  and  any  additional  issues 
in  a  draft  report  at  a  later  date.  We  request  that  you  provide  a  response  to  this 
memorandum  by  June  8,1999.  If  there  are  any  questions,  please  contact  Ms.  Kimberley 
Caprio,  Program  Director  at  (703)  604-9139  or  DSN  664-9139. 

F.  Jay  Lane 
Director 

Finance  and  Accounting  Directorate 


17 


DFAS  has  made  significant  progress  in  addressing  testing  requiremenis  for  its 
funcUooal  areas  including  the  issuance  of  a  Master  Plan,  identification  of  levels  of 
responsibility .  and  checklists  for  test  planning  purposes.  During  our  review,  we 
tdeniified  the  following  concerns  that  should  be  addressed  by  DFAS.  On  May  27 , 

1999,  we  met  with  DFAS  officials  to  discuss  the  concerns  and  actions  to  be  taken. 

1.  Roles  and  Responsibilities.  ITie  DFAS  Master  Plan  identified  four  levels  of 
responsibility  for  end  to  end  testing  including  Headquarters  fiinctionaJ  proponents, 
systems  managers,  event  leaders,  and  thread  leaders.  The  Plan  defined  roles  and 
responsibilities  for  functional  proponents  and  systems  managers,  but  did  not  provide 
details  on  the  responsibilities  for  either  the  event  or  thread  leaders  During  the  May 
27,  1999  meeting,  DFAS  Headquarters  personnel,  acknowledged  tlie  need  for  oversight 
and  agreed  to  provide  the  details  immediately. 

In  addition,  the  Master  Plan  was  not  issued  until  May  11, 1999,  and  in  some  cases  had 
not  arrived  at  the  event  leader  level  until  May  18,  1999.  However,  functional  event 
plans  and  allocation  of  responsibilities  was  alreacfy  occurring.  As  a  result,  the 
individuals  delegated  the  responsibilities  may  not  have  been  appropriate.  For  example, 
for  the  Travel  pay  event,  the  same  person  was  tasked  as  botii  the  fonctional  proponent 
and  the  event  leader  As  a  result,  it  precludes  the  separation  of  duties  by  allowing  one 
function  to  oversee  the  other  function  To  ensure  that  tiie  4  levels  of  responsibility  are 
appropt  lately  staffed,  the  DFAS  Headquarters  Project  Office  should  review  the 
assigned  personnel  and  ensure  that  they  arc  aware  and  understand  their  delegated 
responsibilities. 

2,  Master  Plan  Checklists.  The  DFAS  Y2K  Master  Plan  included  four  checklists 
to  be  used  by  DFAS  Headquarters  personnel,  the  functional  area  proponent,  the  event 
leader,  and  the  tester.  These  checklists  require  DFAS  personnel  to  assess  the 
effectiveness  of  the  end-to-end  testing  program  at  each  designated  level  including  such 
items  as  assessing  the  adequacy  of  testing  staff,  funds,  and  interface  agreements  The 
DFAS  Master  Plan  stated  tiiat  these  checklists  would  "piovide  independent  auditors 
with  evidence  of  con^liancc  with  the  end-to-end  test  requirements,**  however,  the 
Master  Plan  did  not  make  completion  of  the  checklists  mandatory. 

We  believe  the  checklists  should  be  mandatory  and  maintained  at  the  functional  level 
along  with  tesi  results.  The  2  to  3  page  checklists  provide  an  excellent  means  to 
ensure  and  document  that  essential  steps  were  taken  prior  to  performing  end  to  end 
testing  of  DFAS  functional  areas.  Completion  of  the  planning  section  of  these 
checklists  provides  a  tool  to  help  ensure  compliance  with  the  Master  Plan  requirements 
and  allow  fot  early  corrections  of  deviations  or  omissions  from  the  plan.  FUrther,  use 
of  the  checklist  affords  standardization  of  the  process  used  throughout  DFAS  for  end 
to  end  test  planning  efforts.  Without  the  use  of  the  checklists,  DFAS  lacks  assurance 
that  the  testing  was  complete,  adequate,  and  consistent.  We  also  believe  that  a 
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signature  block  or  notation  should  be  included  in  the  checklists  to  establish 
accountabilio^  for  the  responses  and  to  facilitate  quick  actions  should  a  problem  arise 
later. 

3.  Interfacing  Systems.  DFAS  relies  heavily  on  interfacing  systems  to  provide 
the  majority  of  data  included  in  DFAS  systems.  As  such,  coordination  and 
compatibility  of  data  exchanged  with  interfacing  systems  is  critical  to  ensuring 
successful  Y2K  end  to  end  tests  If  data  from  a  non-compliant  system  feeds  into  a 
DFAS  system,  the  potential  exists  for  the  DFAS  system  to  not  be  able  to  ftmetion 
properly  after  Y2K  The  level  of  assurance  being  obtained  by  DFAS  functional  area 
officials  regarding  Y2K  compliance  of  interfacing  systems  varies  from  exchanging 
documentation  to  merely  assuming  that  interfacing  systems  are  compliant  or  relying  on 
verbal  responses. 

Given  the  significant  potential  impact  of  interfaces  on  successful  testing,  we  believe 
that  DFAS  functional  leaders  should  take  the  extra  step  to  validate  that  key  interfacing 
systems  are.  in  fact,  compliant.  Information  on  the  compliance  of  each  DoD  mission 
critical  system  should  be  available  in  the  OSD  database.  As  such.  DFAS  personnel  for 
the  functional  areas  should  be  able  to  access  the  database  and  validate  that  those 
applicable  interfacing  partners  are  clearly  designated  as  Y2K  compliant  before  entering 
the  end-to-end  test.  We  discussed  this  matter  with  DFAS  Headquarters  officials  who 
agreed  that,  while  they  are  only  testing  with  compliant  interfacing  partners,  it  is 
reasonable  that  DFAS  review  the  daubase  to  ensure  that  interfacing  partners  are 
compliant  prioi  to  testing. 

4.  Critical  Crossover  Dates.  The  DFAS  Master  Plan  identified  5  critical  cross¬ 
over  dates  as  mandatory  for  inclusion  during  end-to-end  tests.  The  dates  are  consistent 
with  the  5  dates  recommended  by  the  Assistant  Secretary  Defense  (Command,  Control, 
Communications,  and  Intelligence)  The  dates  are  fiscal  year  2000.  calendar  year 

2000.  leap  year  crossing  (February  29,  2000),  fiscal  year  2001 ,  and  calendar  year 

2001 .  Developers  of  DFAS  test  plans  have  hot  planned  to  test  all  5  dates.  For 
example,  the  Marine  Corp  Total  Force  System  is  only  testing  the  leap  year  2000 
crossover.  The  Computerized  Accounts  Payable  System' is  not  testing  the  fiscal  year 
and  the  calendar  year  2001  crossovers  The  reduced  number  of  dates  being  tested  is  a 
result  of: 

•  The  test  plans  being  developed  prior  to  the  issuance  of  the  DFAS  Master  Plan  on 
May  11,  1999, 

•  personnel  pay  systems,  for  example,  not  being  Impacted  by  fi.scal  yc^  changes, 

•  Funding  being  allocated  based  on  test  plans  developed  prior  to  the  Master  Plan. 

In  order  to  ensure  compatibility  of  interfacing  systems,  it  is  important  that  the  same 
dates  are  tested,  particularly  where  DFAS  systems  feed  data  to  other  systems.  For 
example,  data  from  systems  within  the  Travel  Pay  test  event  feed  into  systems  within 
the  Disbursing  test  event.  Further,  once  processed  within  disbursing,  data  is  fed  to  both 
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accounting  and  back  to  travel  systems.  Incoming  files  to  the  Automated  Disbursing 
System  (ADS)  (part  of  the  Disbursing  test  event)  plan  to  test  all  dates  specified  in  the 
DFAS  Y2K  Master  Plan  However,  the  Travel  Pay  test  event  does  not  plan  to  test  the 
fiscal  year  2000  to  2001  crossover.  As  a  result,  the  potential  exists  that  data  relying  on 
the  fiscal  year  2000  to  2001  crossover  may  not  funaion  properly.  Meanwhile,  DFAS 
may  report  a  successful  test  based  on  the  less  than  5  dates  being  tested. 

It  is  important  that  interfacing  systems  select  test  dates  in  a  similar  manner  to  ensure 
Y2K  data  flows  through  each  system  appropriately.  DFAS  functional  leaders  should 
ensure  that  critical  crossover  dates  for  each  of  the  seven  functional  areas  are  compatible 
prior  to  testing. 

5,  Data  Collection  and  Analysis,  The  DoD  Y2K  Manageincnt  Plan  states  that 
Y2K  event  output  products  such  as  plans  and  piocedures  should  specify  in  detail  what 
data  needs  to  be  collected,  who  will  analyze  the  data,  and  how  it  will  be  an^yzed. 
Essentially,  the  requirement  is  to  define  expected  test  results.  Consistent  with  the  DoD 
Plan,  the  DFAS  Master  Plan  requires,  as  exit  criteria  to  the  test  planning  phase,  that 
responsible  parties  specify  pass/fail  criteria  for  all  tests,  that  data  collection  procedures 
ore  in  place,  and  mechanisms  needed  to  capture  data  are  installed.  The  DFAS  Master 
Plan,  however,  does  not  specify: 

•  What  types  of  data  should  be  collected  to  ensure  consistency  In  reporting  test 
results. 

•  A  methodology  for  each  DFAS  organization  to  document  the  data  collection 
process  in  the  appropriate  Event  Plan 

For  tile  7  DFAS  fiinctional  evente,  data  collection  and  data  analysis  plans  arc 
cither  nonexistent  or  do  not  ensure  the  tests  will  be  judged  objectively  For  example , 
the  Defense  Industrial  Financial  Management  System  (DIFMS)  Test  Plan,  which  is  part 
of  the  Accounting  Test  Event,  plans  to  review  reports,  queried  data,  and  DIFMS 
screens  to  accomplish  data  analysis,  but  did  not  establish  expected  test  results  criteria  or 
a  baseline  that  could  be  used  to  determine  the  adequacy  or  accuracy  of  the  reports, 
queries,  and  screens.  As  another  example,  the  Civilian  Pay  Event  lacks  either  a  data 
collection  plan  or  a  data  analysis  plan.  Instead,  the  Event  Leader  indicated  that  years  of 
prior  testing  and  DCPS  experience  will  identify  discrepancies  should  they  arise. 

Both  DoD  and  DFAS  require  the  establishment  of  a  structured  approach  to  testing 
including  identifying  expected  outcomes,  test  participants,  and  other  details.  Without 
such  plans,  there  is  no  organized  or  standardized  approach  between  the  participating 
systems,  nor  any  assurance  that  test  goals  arc  met  and  tests  were  successful.  Given  the 
nature  of  end-to-end  testing,  with  its  large  numbers  of  participating  or  ^partner'' 
systems,  it  is  prudent  to  ensure  that  the  data  collection  is  as  consistent  as  possible  for 
each  event,  and  that  the  analysis  of  the  test  data  is  objective.  Without  the  definition  of 
data  collection  and  data  analysis  plans  before  testing  begins,  this  will  be  difficult. 
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Functional  Test  Leaders  need  to  ensure  ihat  detailed  test  collection,  results,  and 
analysis  Tcquiremcnts  are  clearly  defined  prior  to  testing. 

5,  Transportation  Pay  Event.  DFAS  identified  Transporiation  Pay  as  one  of  the 
7  functional  areas  for  testing  purposes  However,  t)FAS  has  not  yet  developed  an  end- 
to-end  test  plan  for  tfie  event.  Thcie  arc  two  systems  involved  in  transportation,  the 
Defense  Transportation  Pay  System  (DTRS)  and  the  Military  Traffic  Management 
Comniand  -  Financial  Management  System  (MTMC-FMS).  'fhe  Transportatioa  Pay 
Event  Leader  stated  that  MTMC-FMS  testing  during  Y2K  conversion  process 
accomplished  the  end-to-end  requirements  of  the  Master  Plan.  DFAS  has  subsequently 
contracted  with  the  Joint  fiiteroperability  Testing  Command  (JITC)  to  independently 
verify  and  validate  the  prior  testing.  We  plan  to  follow-up  on  this  functiomil  area 
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Appendix  D.  Defense  Finance  and  Accounting 

Service  Interim  Report  Response 


DFAS-HQ/S 


DEFENSE  FINANCE  AND  ACCOUNTING  SERVICE 


1«31  JEFFERSON  DAVIS  HIGHWAY 
ARLINGTON,  VA  «2aAa-Sa»l 

JUN  8  1999 


MEMORANDUM  FOR  DIRECTOR,  FINANCE  AND  ACCOUNTING  DIRECTORATE 

OFFICE  OF  THE  INSPECTOR  GENERAL,  DEPARTMENT  OF 
DEFENSE 


SUBJECT;  Status  of  Audits  of  Financial  Functional  Area 
year  2000  End-to-End  Tests 

The  attached  outlines  Defense  Finance  and  Accounting 
Service  (DFAS)  response  to  the  DoD  Inspector  General's  (IG) 
initial  review  of  and  concerns  about  DFAS'  End-to-End  Test  (E2E) 
Plans,  DFAS  recognizes  that  a  great  deal  of  work  is  still  to  be 
done  to  ensure  all  necessary  requirements  for  E2E  are 
accomplished.  To  meet  this  goal,  DFAS  has  conducted  meetings 
with  event  and  thread  leaders  to  review  all  E2E  guidelines  and 
requirements . 

All  conceriis  addressed  in  the  DoD  IG' s  memo  are  being 
addressed. 

Roles  and  Responsibilities;  Concur.  Action  to  expand 
event  and  thread  leader  roles  will  be  accomplished  by 
June  25,  1999. 

Master  Plan  Checklist;  Non-concur,  DFAS  will  not  mandate 
the  checklist. 

Interfacing  Systems:  concur.  This  action  is  considered 
completed,  but  with  periodic  updates. 

Critical  Crossover  Dates:  Concur.  This  action  Is 
completed. 

Data  Collection  and  Analysis:  Concur.  This  is  an  ongoing 
action  with  no  specific  target  date. 

Transportation  Pay  Event;  Concur.  This  is  an  ongoing 
action  with  a  target  completion  date  of  June  30,  1999. 


C.  Vance  Kauzlarich 
Director  for  Information  and  Technology 


Attachment : 
As  Stated 


22 


Roles  and  Rasponslbllltiaa.  Concur.  DFAS  acknowledges  that 
testing  and  planning  activities  were  occurring  prior  to  the 
issuance  of  the  Master  Plan  on  6  May,  However,  several 
coordination  meetings  had  already  occurred  and  guidance  on 
developing  Kvent  Plans  was  issued  on  31  March.  DFAS  also 
acknowledges  the  need  to  clarify  and  augment  the  roles  and 
responsibilities  of  the  event  and  thread  leaders  in  the 
DPAG  E2E  Master  Plan.  We  are  currently  making  site  visits 
and  meeting  with  the  testing  teams  to  clarify  roles  and 
responsibilities  and  are  updating  the  Master  plan  as  well. 
It  should  be  noted  there  may  be  an  overlap  in  the  area  of 
responsibility,  due  to  the  fact  that  the  internal  DFAS 
support  structure  for  each  business  pxocess/application  has 
a  great  bearing  upon  the  specific  breakout  of  roles  and 
responsibilities.  DFAS  does  not  view  this  as  a  conflict  or 
an  inappropriate  assignment  of  duties. 

Master  Plan  Checklist.  Non-concur,  DFAS  designed  and 
issued  these  checklists  as  tools  to  assist  DFAS  personnel 
responsible  for  planning,  tracking,  and  conducting  end  to 
end  testing.  Because  each  business  area/application  has  a 
normal  testing  practice  already  established,  DFAS  did  not 
make  the  checklists  mandatory,  and  would  prefer  t©  keep  the 
use  of  checklists  optional.  However,  DFAS  will  encourage 
the  use  of  the  checklists  whenever  possible. 

Interfacing  Systems,  Concur.  DFAS  agrees  that  coordination 
and  compatibility  of  data  exchange  between  DFAS  systems  and 
their  interface  partners  is  essential  to  a  successful  Y2K 
effort.  DFAS  has  pursued  this  goal  for  the  past  two  years, 
DFAS  has  established  Interface  Agreements  with  all  of  its 
interface  partners.  This  effort  generated  in  excess  of 
1400  agreements.  In  addition,  DFAS  has  tracked  and  updated 
on  a  monthly  basis  the  status  of  testing  and  compliance  of 
each  of  its  interface  partners.  DFAS  system  mangers  are 
well  aware  of  the  status  of  each  of  its  partners.  DFAS 
will  continue  to  track  and  monitor  the  status  of  its 
interface  partners  mission  critical  and  other, 

Critioal  Crossover  Dates*  Concur.  DFAS  acknowledges  the 
importance  of  testing  as  many  dates  as  possible,  and  the 
coordination  of  these  dates  among  partners.  All  DFAS 
managers  have  been  encouraged  to  coordinate  this  initiative 
with  all  pertinent  parties.  It  must  be  understood  that 
dates  do  not  necessarily  play  an  important  part  in  the 
relationship  of  one  system  to  another.  The  DFAS  E2E  Master 
Plan  has  recently  been  updated  to  empower  Lhe  Functional 
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Managers  with  determining  which  dates  arc  critical  for 
testing  within  their  specific  business  process.  We  have 
also  hired  JITC  to  Independently  validate  and  verify  our 
planning  efforts . 

Data  Collaotion  and  Analysis.  Concur.  DFAS  agrees  that 
current  plans  lack  specific  exit  criteria  and  we  are  taking 
action  to  strengthen  this  area  of  our  plana.  DFAS  8000.1“ 
Part  E,  Chapter  3,  Test  and  Evaluation  provides  guidance 
concerning  data  collection  and  analysis.  Our  central 
design  activities  normally  plan  and  execute  their  tests, 
using  this  guidance,  precluding  the  need  for  specific 
guidance  to  be  issued  relative  to  E2E  testing. 

Each  testing  agent  within  DFAS  implements  the  regulation 
within  their  own  construct,  resulting  in  a  non-'Standard, 
but  successful,  data  collection  and  analysis  process. 
Because  Y2K  E2E  testing  requirements  are  not  system 
centric,  but  business  process  centric,  we  have  hired  JITC 
to  independently  validate  and  verify  our  E2E  planning  and 
testing  efforts.  The  JITC  analysis/evaluation  will 
document  specific  risks  associated  with  data  collection  and 
analysis  procedures,  in  sufficient  time  for  us  to  take 
corrective  action. 

Another  measure  of  risk  mitigation  is  to  conduct  site 
visits,  where  we  meet  with  all  thread  participants.  During 
these  meetings  we  are  emphasizing  the  need  for  adequate 
documentation  of  their  data  collection  and  analysis 
procedures,  as  well  as  documenting  version  control  and 
configuration  management  procedures. 

Transportation  Pay  Event.  Concur,  In  addition  to  the 
Transportation  Pay  System,  DFAS  has  identified  several 
other  sy. stems,  which  claim  completion  of  the  end  to  end 
testing  initiative,  JITC  will  be  used  to  verify  that  these 
systems  indeed  have  met  E2E  requirements.  If  any  system 
fails  to  pass  the  validation  of  JITC,  steps  will  be 
initiated  to  complete  all  or  any  portion  of  the  E2E  process 
that  needs  to  be  completed. 
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Appendix  E.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 
Deputy  Chief  Information  Officer  and  Deputy  Assistant  Secretary  of  Defense  (Chief 
Information  Officer  Policy  and  Implementation) 

Principal  Director  for  Year  2000 


Joint  Staff 

Director,  Joint  Staff 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 
Chief  Information  Officer,  Department  of  the  Army 
Inspector  General,  Department  of  the  Army 
Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 

Chief  Information  Officer,  Department  of  the  Navy 

Inspector  General,  Department  of  the  Navy 

Auditor  General,  Department  of  the  Navy 

Inspector  General,  Marine  Corps 

Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Chief  Information  Officer,  Department  of  the  Air  Force 
Inspector  General,  Department  of  the  Air  Force 
Auditor  General,  Department  of  the  Air  Force 


other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Finance  and  Accounting  Service 
Director,  Defense  Information  Systems  Agency 

Inspector  General,  Defense  Information  Systems  Agency 
United  Kingdom  Liaison  Officer,  Defense  Information  Systems  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
Director,  Defense  Systems  Management  College 


Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 

Office  of  Information  and  Regulatory  Affairs 
General  Accounting  Office 

National  Security  and  International  Affairs  Division 
Technical  Information  Center 

Director,  Defense  Information  and  Financial  Management  Systems,  Accounting  and 
Information  Management  Division,  General  Accounting  Office 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Committee  on  Governmental  Affairs 

Senate  Special  Committee  on  the  Year  2000  Technology  Problem 

House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Reform 

House  Subcommittee  on  Government  Management,  Information,  and  Technology, 
Committee  on  Government  Reform 

House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  Technology,  Committee  on  Science 
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Defense  Finance  and  Accounting  Service 
Comments 


DFAS~HQ/S 


DEFENSE  FINANCE  AND  ACCOUNTING  SERVICE 

1031  JKPFCRSON  DAVIS  HIGHWAY 
ARLINGTON.  VA  22240.S29  I 


SEP  3  0  1999 


MEMORANDUM  FOK  DIRECTOR,  FINANCE  AND  ACCOUNTING  DIRECTORATE 
OFnCE  OF  THE  INSPECTOR  GENERAL.  DEPARTMENT  OF 
DEFENSE 

SUBJECT:  Draft  of  a  Proposed  Audit  Report  on  Defense  Disbursing  Year  2000 
End-to-End  Testing  (Project  No.  9FG-9.029)  dated  September  2, 1999 


The  Defense  Finfince  and  Accounting  Service  response  to  the  Department  of  Defense 
Inspector  General’s  Report  on  Defense  Disbursing  Year  2000  End-to-End  Testing  is  attached . 

The  recommendations  in  subject  report  arc  being  implemented  as  outlined.  The 
Disbursing  End-to-End  Testing  has  been  fully  integrated  with  all  entitlement  business  areas  All 
disbursing  systems  have  been  included  in  numerous  successful  tests  to  date. 

C.  Vance  Kauzlaris^ 

Director  for  Information  and  Technology 
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As  stated 


COMMENTS  ON 
DRAFT  REPORT  OF  AUDIT 
ON  DISBURSING 

YEAR  2000  END-TO-END  TESTING 
PROJECT  NO.  9FG-9029 


1.  Recommendation  #1:  Initiate  steps  to  Include  the  Standard  Negotiable  Items 
Processing  System  (SNIPS)  In  the  end-to-end  testing  process,  whether  as  part  of  the 
disbursing  event  or  another  event. 

Concur. 

We  have  Included  SNIPS  In  the  Military  Pay  End-lo-End  Test  Plan.  DFAS  Indianapolis 
Center  and  DFAS  Denver  Center  have  agreed  to  execute  an  Integrated  test  scenarii.), 
entitled  Military  Pay  (Army  Payroll  Check),  which  Includes  receipt  of  a  check  payment 
file  from  the  Military  Pay  system  (DJMS)  and  Issuance  of  a  check  and  report  data 

Estimated  test  completion  dale:  September  30. 1999 

2.  Recommendation  #2:  Initiate  alternative  measures  to  mitigate  the  risks  of 
disbursing  systems  not  being  able  to  successfully  process  data  after  the  year  2000. 
Such  alternative  measures  may  include,  for  example,  performing  supplementary  encl-to- 
end  tests  of  the  event,  using  code  scanners,  or  expanding  event  contingency  plans. 

Concur. 

The  disbursing  contingency  plan  Is  in  place  and  has  been  successfully  tested. 

DFAS  will  do  code  scanning  of  its  systems  with  the  Joint  Interoperability  Technology 
Command  and  the  Systems  Support  Group  at  Gunter  Air  Force  Base.  Code  scanning 
will  be  accomplished  on  all  disbursing  systems,  to  Include  Standard  Finance  System - 
Redesign  (SRD-1),  Automated  Disbursing  System  (ADS),  Standard  Negotiable 
Instrument  Processing  System  (SNIPS)  and  Integrated  Payment  ar>d  Collection  Sys  em 
(IPC). 


Estimated  completion  date  to  code  scan  all  four  systems:  October  30. 1999 
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Audit  Team  Members 

The  Finance  and  Accounting  Directorate,  Office  of  the  Assistant  Inspector 
General  for  Auditing,  DoD,  prepared  this  report, 

F.  Jay  Lane 
Salvatore  D.  Guli 
Kimberley  A.  Caprio 
Dennis  M.  Conway 
Jacqueline  J.  Vos 
Robyn  N.  Stanley 
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